• See here for AOP spectra from PRLA buoy location
  • This notebook summarizes cholorphyll, DOM, and suspended matter data from sampling and sensors. These are inputs to forward model or reponse for statistical models.
  • Sampling data is chl a, pheophytin, DOC, TOC, UV abs 254 and 280, Fe, TSS, TDS
  • Data for sample collected closest in time to each flight date is reported, usually one value for each flight year
  • Sensor data is chl, fDOM, turbidity, uv abs 254 (from SUNA), uv abs 350 (from SUNA)
  • Closet in time sensor values are from a moving average of sensor time series
  • Sensor data at time of ground based sample collection is compared for chl, fDOM vs DOC, fDOM vs uv254, TSS vs turbidity, uv 254
  • L0 SUNA data needs to be downloaded and processed separately in processing project
  • Updating the AOS and AIS data is handled separately in neon-aqu-aos.
  • AOP flight times are determined from L1 radiance metadata.

Summary of data overlap for PRLA

  • Green flightlines from multiple dates in 2019 (July 26, 27) and 2020 (June 24, July 2)
  • In 2019, chl sampling from July 11 (2 weeks before), chemistry August 6 (11 days after), and sensor data is available
  • In 2020, chl sampling from June 15 (9 days before), chemistry data not all available from closest sampling date (only TOC not DOC). sensor data available for first flight date but not later date (which had a higher spectra with more pronounced peaks). The chl sample from the next bout was > 100 ug/L
  • Also AOP data collected in 2016 and 2017 under yellow conditions but no ground-based data
mysite = 'PRLA'

I. Chlorophyll

I.(A) AOS Sampling

  • From processed chl a files on Box
  • reported in ug/L
  • seasonal sampling, 2 analytes

See how many locations chl data is collected from

See how many locations chl data is collected from (variable Y axis)

Availability of chlorophyll AOS data for each flight date

  • Default threshold 1 = 30 days
  • Default threshold 2 = 60 days
flightdate aos_before aos_after days_before days_after min_days meets_thresh1 meets_thresh2
2016-06-29 NA 2019-07-11 NA 1107 1107 FALSE FALSE
2017-06-21 NA 2019-07-11 NA 750 750 FALSE FALSE
2017-06-26 NA 2019-07-11 NA 745 745 FALSE FALSE
2019-07-26 2019-07-11 2019-09-10 15 46 15 TRUE TRUE
2019-07-27 2019-07-11 2019-09-10 16 45 16 TRUE TRUE
2019-07-30 2019-07-11 2019-09-10 19 42 19 TRUE TRUE
2020-06-24 2020-06-15 2020-09-29 9 97 9 TRUE TRUE
2020-06-26 2020-06-15 2020-09-29 11 95 11 TRUE TRUE
2020-07-02 2020-06-15 2020-09-29 17 89 17 TRUE TRUE
flightdate aos_match days
2019-07-26 2019-07-11 15
2019-07-27 2019-07-11 16
2019-07-30 2019-07-11 19
2020-06-24 2020-06-15 9
2020-06-26 2020-06-15 11
2020-07-02 2020-06-15 17
  • Make sure plant algae lab units are all micrograms per liter
  • Check sample conditions
## [1] "microgramsPerLiter"
## [1] "condition ok"

chl AOS x flightdates

These are the closest chl AOS values for each flight date

aos_match flightdate days pheophytin chlorophyll a
2019-07-11 2019-07-26 15 3.01 2.31
2019-07-11 2019-07-27 16 3.01 2.31
2019-07-11 2019-07-30 19 3.01 2.31
2020-06-15 2020-06-24 9 4.20 3.45
2020-06-15 2020-06-26 11 4.20 3.45
2020-06-15 2020-07-02 17 4.20 3.45

Algal Community

Algal types include:

  • Bacillariophyceae
  • Cyanobacteria
  • Chlorophyta and Charophyta
  • Xanthophyceae
  • Unclassified

Algal divisions include:

  • Chrysophyta
  • Cyanophyta
  • Chlorophyta

Algal classes include:

  • Bacillariophyceae
  • Myxophyceae
  • Chlorophyceae
  • Chrysophyceae

Abundance is reported in a range of units:

  • number of cells, natural units, or valves (Chrysophyta/Bacillariophyceae)
  • cell density
  • biovolume density

Starting in 2018 algal type no longer reported?

## 
## ── Column specification ────────────────────────────────────────────────────────
## cols(
##   namedLocation = col_character(),
##   collectDate = col_date(format = ""),
##   sampleID = col_character(),
##   algalParameter = col_character(),
##   class = col_character(),
##   sum_param = col_double(),
##   rel_abun = col_double()
## )
## 
## ── Column specification ────────────────────────────────────────────────────────
## cols(
##   namedLocation = col_character(),
##   collectDate = col_date(format = ""),
##   sampleID = col_character(),
##   algalParameter = col_character(),
##   division = col_character(),
##   sum_param = col_double(),
##   rel_abun = col_double()
## )
## 
## ── Column specification ────────────────────────────────────────────────────────
## cols(
##   namedLocation = col_character(),
##   collectDate = col_date(format = ""),
##   sampleID = col_character(),
##   algalParameter = col_character(),
##   algalType = col_character(),
##   sum_param = col_double(),
##   rel_abun = col_double()
## )
## Warning: Removed 1 rows containing missing values (position_stack).

## Warning: Removed 1 rows containing missing values (position_stack).

## Warning: Removed 1 rows containing missing values (position_stack).

I.(B) Chl AIS time series

The EXO total algae sensor is a dual‐channel fluorometer that uses a 470nm excitation beam that excites chlorophyll a and a second 590 nm excitation beam that excites the phyocyanin accessory pigment found in blue‐green algae (cyanobacteria). Chlorophyll concentration is a biogeochemically relavant parameter that is readily available by remote sensing and can be can serve as a proxy for phytoplankton biomass and light attenuation (Oestreich et al., 2016, Ganju et al., 2014, Jaud et al., 2012)

Sensor chl data availability

flightline_datetime check_30day check_10day check_1day check_12hr
2016-06-29 15:45:47 FALSE FALSE FALSE FALSE
2016-06-29 15:52:09 FALSE FALSE FALSE FALSE
2016-06-29 17:22:24 FALSE FALSE FALSE FALSE
2017-06-21 20:58:28 FALSE FALSE FALSE FALSE
2017-06-26 15:33:21 FALSE FALSE FALSE FALSE
2017-06-26 16:04:51 FALSE FALSE FALSE FALSE
2017-06-26 16:11:11 FALSE FALSE FALSE FALSE
2019-07-26 15:20:23 TRUE TRUE TRUE TRUE
2019-07-26 16:22:11 TRUE TRUE TRUE TRUE
2019-07-26 22:09:25 TRUE TRUE TRUE TRUE
2019-07-27 15:27:04 TRUE TRUE TRUE TRUE
2019-07-30 15:34:20 TRUE TRUE TRUE TRUE
2020-06-24 17:36:27 TRUE TRUE TRUE TRUE
2020-06-24 16:26:15 TRUE TRUE TRUE TRUE
2020-06-24 16:18:44 TRUE TRUE TRUE TRUE
2020-06-26 19:28:06 TRUE TRUE TRUE FALSE
2020-07-02 17:14:10 TRUE TRUE FALSE FALSE

2019 chl AIS plots

  • Filtered to only 0 quality flags

2020 chl AIS plots

  • No QF filters on these plots
  • Is QF because of lack of calibration?

2019 Chl avg

  • Smooth the chl time series using a rolling mean with zoo::rollapply.
  • First convert to a regular 5 minute time series using tsibble

Get the moving average value closet to flight time.

  • 5min is the original data from that row
  • ma01 = 1 hour window, ma03 = 3 hour window, and so on
  • chl_ma04u is mean, the others are all mean
collectDateTime datetime chl5min chl_ma01 chl_ma03 chl_ma04 chl_ma04u chl_ma06 chl_ma12
2019-07-26 15:20:23 2019-07-26 15:20:00 5.83 6.245 6.285 6.290 6.312174 6.350 6.610
2019-07-26 16:22:11 2019-07-26 16:20:00 6.70 6.175 6.285 6.310 6.327500 6.425 6.610
2019-07-26 22:09:25 2019-07-26 22:10:00 7.47 7.665 7.735 7.695 7.881042 7.660 7.650
2019-07-27 15:27:04 2019-07-27 15:25:00 8.16 8.195 8.380 8.400 8.477083 8.455 8.655
2019-07-30 15:34:20 2019-07-30 15:35:00 18.12 17.850 17.460 17.460 17.539130 17.860 19.670

2020 Chl avg

  • Smooth the chl time series using a rolling mean with zoo::rollapply.
  • First convert to a regular 5 minute time series using tsibble
  • Using median for 2020 because of QA issues? (or could remove outliers before calculating running median)

Get the moving average value closet to flight time

collectDateTime datetime chl5min chl_ma01 chl_ma03 chl_ma04 chl_ma04u chl_ma06 chl_ma12
2020-06-24 17:36:27 2020-06-24 17:35:00 2.54 2.320 2.48 2.545 2.638333 2.715 3.150
2020-06-24 16:26:15 2020-06-24 16:25:00 2.74 2.565 2.56 2.565 2.581875 2.680 3.165
2020-06-24 16:18:44 2020-06-24 16:20:00 2.54 2.565 2.56 2.585 2.587917 2.680 3.175
2020-06-26 19:28:06 2020-06-26 19:30:00 NA NA NA NA NaN NA NA
2020-07-02 17:14:10 2020-07-02 17:15:00 NA NA NA NA NaN NA NA

I.(C) Chl AIS vs AOS

  • use chla_df date and times with the aop/ais matching function to determine which sampling dates have sensor data from same day around time of sampling.

already sort of done for suna data… follow this pattern

  • Get chl sensor data from all AOS sampling dates (use ais with buffer 2 fucntion?)
  • show what is available
  • calculate rolling averages of chl time series
  • get rolling average values for time closest to sampling using which min on difftime
  • little overlap because of seasonal removal of buoys
aos_datetime check_3day check_1day check_6hr check_1hr
2019-07-11 13:53:00 TRUE TRUE TRUE TRUE
2019-09-10 13:37:00 TRUE TRUE TRUE TRUE
2020-06-15 15:09:00 TRUE TRUE TRUE TRUE
2020-09-29 13:21:00 FALSE FALSE FALSE FALSE
2021-04-20 14:40:00 FALSE FALSE FALSE FALSE
aos_datetime check_3day check_1day check_6hr check_1hr
2019-07-11 13:53:00 TRUE TRUE TRUE TRUE
2019-09-10 13:37:00 TRUE TRUE TRUE TRUE
2020-06-15 15:09:00 TRUE TRUE TRUE TRUE

collect_date pheophytin chla chl_tot
2019-07-11 3.01 2.31 5.32
2019-09-10 3.08 2.79 5.87
2020-06-15 4.20 3.45 7.65
2020-09-29 72.40 124.44 196.84
2021-04-20 21.38 20.03 41.41

Compare AOS and AIS

II. DOM

II.(A) AOS Sampling

  • From processed Surface water chemistry files on Box

EPA Method description:

  • The UVA procedure requires that the sample be passed through a 0.45 um filter and transferred to quartz cell. It is then placed in a spectrophotometer to measure the UV absorbance at 254 nm and reported in cm -1.

  • The SUVA procedure requires both the DOC and UVA measurement. The SUVA is then calculated by dividing the UV absorbance of the sample (in cm -1) by the DOC of the sample (in mg/L) and then multiplying by 100 cm/M. SUVA is reported in units of L/mg-M. The formula for the SUVA may be found in Section 12.2.

Ignoring data before 2016

## Warning: Removed 2 rows containing missing values (geom_point).

Correct older UV Absorbance wavelengths from 250 to 254, assuming all 250 nm should be 254.

swchem_site_df <- swchem_site_df %>%
  dplyr::mutate(analyte = dplyr::case_when(analyte == 'UV Absorbance (250 nm)' ~
                                             'UV Absorbance (254 nm)', TRUE ~ analyte))
## Warning: Removed 2 rows containing missing values (geom_point).

How much does SUVA 254/280 ratio change?

How much does organic carbon dissolved proportion change?

Adjusts sample IDs in a new column to group together raw and filtered, while keeping track of duplicates

## Warning: Removed 11 rows containing missing values (geom_point).

Match DOM AOS and AOP data

Need to do this separately for each analyte because they arent always all reported.

DOC x flightdates

These are the closest DOC values (mg/L) for each flight date

  • A list of values means there are duplicate samples (multiple sampleIDs) from same date.
flightdate aos_before aos_after days_before days_after min_days meets_thresh1 meets_thresh2
2016-06-29 2014-06-19 2016-10-19 741 112 112 FALSE FALSE
2017-06-21 2017-06-08 2017-07-06 13 15 13 TRUE TRUE
2017-06-26 2017-06-08 2017-07-06 18 10 10 TRUE TRUE
2019-07-26 2019-07-01 2019-08-06 25 11 11 TRUE TRUE
2019-07-27 2019-07-01 2019-08-06 26 10 10 TRUE TRUE
2019-07-30 2019-07-01 2019-08-06 29 7 7 TRUE TRUE
2020-06-24 2020-06-09 2020-08-04 15 41 15 TRUE TRUE
2020-06-26 2020-06-09 2020-08-04 17 39 17 TRUE TRUE
2020-07-02 2020-06-09 2020-08-04 23 33 23 TRUE TRUE
flightdate aos_match days
2017-06-21 2017-06-08 13
2017-06-26 2017-07-06 10
2019-07-26 2019-08-06 11
2019-07-27 2019-08-06 10
2019-07-30 2019-08-06 7
2020-06-24 2020-06-09 15
2020-06-26 2020-06-09 17
2020-07-02 2020-06-09 23
aos_match flightdate days DOC
2017-06-08 2017-06-21 13 29.97
2017-07-06 2017-06-26 10 31.68, 31.81, 32.11
2019-08-06 2019-07-26 11 35.01
2019-08-06 2019-07-27 10 35.01
2019-08-06 2019-07-30 7 35.01
2020-06-09 2020-06-24 15 26.72
2020-06-09 2020-06-26 17 26.72
2020-06-09 2020-07-02 23 26.72

UV abs x flightdates

These are the closest UV abs values (per 1 cm) for each flight date

  • A list of values means there are duplicate samples (multiple sampleIDs) from same date.
flightdate aos_before aos_after days_before days_after min_days meets_thresh1 meets_thresh2
2016-06-29 NA 2016-10-19 NA 112 112 FALSE FALSE
2017-06-21 2017-06-08 2017-07-06 13 15 13 TRUE TRUE
2017-06-26 2017-06-08 2017-07-06 18 10 10 TRUE TRUE
2019-07-26 2019-07-01 2019-08-06 25 11 11 TRUE TRUE
2019-07-27 2019-07-01 2019-08-06 26 10 10 TRUE TRUE
2019-07-30 2019-07-01 2019-08-06 29 7 7 TRUE TRUE
2020-06-24 2019-12-04 2020-08-04 203 41 41 FALSE TRUE
2020-06-26 2019-12-04 2020-08-04 205 39 39 FALSE TRUE
2020-07-02 2019-12-04 2020-08-04 211 33 33 FALSE TRUE
flightdate aos_match days
2017-06-21 2017-06-08 13
2017-06-26 2017-07-06 10
2019-07-26 2019-08-06 11
2019-07-27 2019-08-06 10
2019-07-30 2019-08-06 7
2020-06-24 2020-08-04 41
2020-06-26 2020-08-04 39
2020-07-02 2020-08-04 33
aos_match flightdate days UV Absorbance (254 nm)
2017-06-08 2017-06-21 13 0.5578
2017-07-06 2017-06-26 10 0.5566, 0.5710, 0.5653
2019-08-06 2019-07-26 11 0.7538
2019-08-06 2019-07-27 10 0.7538
2019-08-06 2019-07-30 7 0.7538
2020-08-04 2020-06-24 41 0.5646
2020-08-04 2020-06-26 39 0.5646
2020-08-04 2020-07-02 33 0.5646
flightdate aos_before aos_after days_before days_after min_days meets_thresh1 meets_thresh2
2016-06-29 NA 2016-10-19 NA 112 112 FALSE FALSE
2017-06-21 2017-06-08 2017-07-06 13 15 13 TRUE TRUE
2017-06-26 2017-06-08 2017-07-06 18 10 10 TRUE TRUE
2019-07-26 2019-07-01 2019-08-06 25 11 11 TRUE TRUE
2019-07-27 2019-07-01 2019-08-06 26 10 10 TRUE TRUE
2019-07-30 2019-07-01 2019-08-06 29 7 7 TRUE TRUE
2020-06-24 2019-12-04 2020-08-04 203 41 41 FALSE TRUE
2020-06-26 2019-12-04 2020-08-04 205 39 39 FALSE TRUE
2020-07-02 2019-12-04 2020-08-04 211 33 33 FALSE TRUE
flightdate aos_match days
2017-06-21 2017-06-08 13
2017-06-26 2017-07-06 10
2019-07-26 2019-08-06 11
2019-07-27 2019-08-06 10
2019-07-30 2019-08-06 7
2020-06-24 2020-08-04 41
2020-06-26 2020-08-04 39
2020-07-02 2020-08-04 33
aos_match flightdate days UV Absorbance (280 nm)
2017-06-08 2017-06-21 13 0.2876
2017-07-06 2017-06-26 10 0.2887, 0.2961, 0.2950
2019-08-06 2019-07-26 11 0.3852
2019-08-06 2019-07-27 10 0.3852
2019-08-06 2019-07-30 7 0.3852
2020-08-04 2020-06-24 41 0.3118
2020-08-04 2020-06-26 39 0.3118
2020-08-04 2020-07-02 33 0.3118

TOC x flightdates

  • TOC is reported in milligramsPerLiter
flightdate aos_before aos_after days_before days_after min_days meets_thresh1 meets_thresh2
2016-06-29 2014-06-19 2016-10-19 741 112 112 FALSE FALSE
2017-06-21 2017-06-08 2017-07-06 13 15 13 TRUE TRUE
2017-06-26 2017-06-08 2017-07-06 18 10 10 TRUE TRUE
2019-07-26 2019-07-01 2019-08-06 25 11 11 TRUE TRUE
2019-07-27 2019-07-01 2019-08-06 26 10 10 TRUE TRUE
2019-07-30 2019-07-01 2019-08-06 29 7 7 TRUE TRUE
2020-06-24 2020-06-09 2020-06-30 15 6 6 TRUE TRUE
2020-06-26 2020-06-09 2020-06-30 17 4 4 TRUE TRUE
2020-07-02 2020-06-30 2020-08-04 2 33 2 TRUE TRUE
flightdate aos_match days
2017-06-21 2017-06-08 13
2017-06-26 2017-07-06 10
2019-07-26 2019-08-06 11
2019-07-27 2019-08-06 10
2019-07-30 2019-08-06 7
2020-06-24 2020-06-30 6
2020-06-26 2020-06-30 4
2020-07-02 2020-06-30 2
aos_match flightdate days TOC
2017-06-08 2017-06-21 13 34.12
2017-07-06 2017-06-26 10 37.87, 38.11, 38.82
2019-08-06 2019-07-26 11 36.05
2019-08-06 2019-07-27 10 36.05
2019-08-06 2019-07-30 7 36.05
2020-06-30 2020-06-24 6 26.05, 26.49, 26.18
2020-06-30 2020-06-26 4 26.05, 26.49, 26.18
2020-06-30 2020-07-02 2 26.05, 26.49, 26.18

Fe x flightdates

  • Fe is reported in milligramsPerLiter
flightdate aos_before aos_after days_before days_after min_days meets_thresh1 meets_thresh2
2016-06-29 2014-06-19 2016-10-19 741 112 112 FALSE FALSE
2017-06-21 2017-06-08 2017-07-06 13 15 13 TRUE TRUE
2017-06-26 2017-06-08 2017-07-06 18 10 10 TRUE TRUE
2019-07-26 2019-07-01 2019-08-06 25 11 11 TRUE TRUE
2019-07-27 2019-07-01 2019-08-06 26 10 10 TRUE TRUE
2019-07-30 2019-07-01 2019-08-06 29 7 7 TRUE TRUE
2020-06-24 2019-12-04 2021-05-04 203 314 203 FALSE FALSE
2020-06-26 2019-12-04 2021-05-04 205 312 205 FALSE FALSE
2020-07-02 2019-12-04 2021-05-04 211 306 211 FALSE FALSE
flightdate aos_match days
2017-06-21 2017-06-08 13
2017-06-26 2017-07-06 10
2019-07-26 2019-08-06 11
2019-07-27 2019-08-06 10
2019-07-30 2019-08-06 7
aos_match flightdate days Fe
2017-06-08 2017-06-21 13 5e-04
2017-07-06 2017-06-26 10 5e-04, 5e-04, 5e-04
2019-08-06 2019-07-26 11 0.01
2019-08-06 2019-07-27 10 0.01
2019-08-06 2019-07-30 7 0.01

II.(B) fDOM AIS time series

The EXO fDOM sensor is a fluorometer with a single excitation/emission pair (365nm/480nm) used to detect the fluorescent fraction of the chromophoric DOM when exposed to near‐UV light. Because of the impacts of temperature and water column absorbance (from a combination of dissolved and particulate compounds) on these readings corrections must be applied to the calibrated data.

  • availability of processed fDOM time series within different time windows
flightline_datetime check_30day check_10day check_1day check_12hr
2016-06-29 15:45:47 FALSE FALSE FALSE FALSE
2016-06-29 15:52:09 FALSE FALSE FALSE FALSE
2016-06-29 17:22:24 FALSE FALSE FALSE FALSE
2017-06-21 20:58:28 FALSE FALSE FALSE FALSE
2017-06-26 15:33:21 FALSE FALSE FALSE FALSE
2017-06-26 16:04:51 FALSE FALSE FALSE FALSE
2017-06-26 16:11:11 FALSE FALSE FALSE FALSE
2019-07-26 15:20:23 TRUE TRUE TRUE TRUE
2019-07-26 16:22:11 TRUE TRUE TRUE TRUE
2019-07-26 22:09:25 TRUE TRUE TRUE TRUE
2019-07-27 15:27:04 TRUE TRUE TRUE TRUE
2019-07-30 15:34:20 TRUE TRUE TRUE TRUE
2020-06-24 17:36:27 TRUE TRUE TRUE TRUE
2020-06-24 16:26:15 TRUE TRUE TRUE TRUE
2020-06-24 16:18:44 TRUE TRUE TRUE TRUE
2020-06-26 19:28:06 TRUE TRUE TRUE FALSE
2020-07-02 17:14:10 TRUE TRUE FALSE FALSE

2019 fDOM plots

## Warning: Removed 3170 rows containing missing values (geom_point).

## Warning: Removed 21 rows containing missing values (geom_point).

## `geom_smooth()` using method = 'loess' and formula 'y ~ x'

## `geom_smooth()` using method = 'loess' and formula 'y ~ x'

## `geom_smooth()` using method = 'loess' and formula 'y ~ x'

2020 fDOM plots

  • Not filtered on QFs
  • QF because of lack of calibration?
## Warning: Removed 8658 rows containing missing values (geom_point).

## Warning: Removed 2552 rows containing missing values (geom_point).

## Warning: Removed 444 row(s) containing missing values (geom_path).
## Warning: Removed 444 rows containing missing values (geom_point).

## Warning: Removed 4737 rows containing missing values (geom_point).

2019 fDOM avg

  • Smooth the fdom time series using a rolling mean with zoo::rollapply.
  • First convert to a regular 5 minute time series using tsibble

Get the moving average value closet to flight time.

  • 5min is the original data from that row
  • ma01 = 1 hour window, ma03 = 3 hour window, and so on
  • chl_ma04u is mean, the others are all mean
collectDateTime datetime fdom5min ma01 ma03 ma04 ma04u ma06 ma12
2019-07-26 15:20:23 2019-07-26 15:20:00 109.76 109.620 109.620 109.630 109.6151 109.630 109.63
2019-07-26 16:22:11 2019-07-26 16:20:00 109.22 109.260 109.260 109.260 109.2767 109.260 109.26
2019-07-26 22:09:25 2019-07-26 22:10:00 108.23 108.075 108.065 108.065 108.0837 108.070 108.09
2019-07-27 15:27:04 2019-07-27 15:25:00 105.65 105.760 105.745 105.745 105.7490 105.745 105.80
2019-07-30 15:34:20 2019-07-30 15:35:00 100.23 100.380 100.350 100.310 100.2890 100.340 100.35

2020 fDOM avg

  • Smooth the fdom time series using a rolling mean with zoo::rollapply.
  • First convert to a regular 5 minute time series using tsibble

Get the moving average value closet to flight time.

  • 5min is the original data from that row
  • ma01 = 1 hour window, ma03 = 3 hour window, and so on
  • chl_ma04u is mean, the others are all mean
collectDateTime datetime fdom5min ma01 ma03 ma04 ma04u ma06 ma12
2020-06-24 17:36:27 2020-06-24 17:35:00 69.59 69.625 69.700 69.730 69.53479 69.865 70.200
2020-06-24 16:26:15 2020-06-24 16:25:00 70.02 69.855 69.880 69.880 69.94437 69.850 70.105
2020-06-24 16:18:44 2020-06-24 16:20:00 69.73 69.945 69.955 69.955 69.96188 69.850 70.105
2020-06-26 19:28:06 2020-06-26 19:30:00 NA NA NA NA NaN NA NA
2020-07-02 17:14:10 2020-07-02 17:15:00 NA NA NA NA NaN NA NA

II.(C) fDOM vs AOS DOM

  • use swchem_df date and times with the aop/ais matching function to determine which sampling dates have sensor data from same day around time of sampling.

  • Get fDOM sensor data from all AOS sampling dates (use ais with buffer2 function?)

  • show what is available

  • calculate rolling averages of fDOM time series

  • get rolling average values for time closest to sampling using which min on difftime

  • little overlap because of seasonal removal of buoys

aos_datetime check_3day check_1day check_6hr check_1hr
2014-05-14 15:15:00 FALSE FALSE FALSE FALSE
2014-06-19 16:00:00 FALSE FALSE FALSE FALSE
2016-10-19 15:30:00 FALSE FALSE FALSE FALSE
2016-11-16 16:00:00 FALSE FALSE FALSE FALSE
2016-12-19 17:53:00 FALSE FALSE FALSE FALSE
2017-01-18 16:15:00 FALSE FALSE FALSE FALSE
2017-02-13 16:15:00 FALSE FALSE FALSE FALSE
2017-03-22 14:45:00 FALSE FALSE FALSE FALSE
2017-04-12 15:15:00 FALSE FALSE FALSE FALSE
2017-05-03 14:20:00 FALSE FALSE FALSE FALSE
2017-06-08 14:45:00 FALSE FALSE FALSE FALSE
2017-07-06 15:00:00 FALSE FALSE FALSE FALSE
2017-08-03 14:50:00 FALSE FALSE FALSE FALSE
2017-09-06 14:15:00 FALSE FALSE FALSE FALSE
2017-10-19 15:00:00 FALSE FALSE FALSE FALSE
2017-11-01 15:15:00 FALSE FALSE FALSE FALSE
2018-01-04 17:00:00 FALSE FALSE FALSE FALSE
2018-02-13 16:45:00 FALSE FALSE FALSE FALSE
2018-03-13 16:05:00 FALSE FALSE FALSE FALSE
2018-04-17 16:03:00 FALSE FALSE FALSE FALSE
2018-05-01 16:26:00 FALSE FALSE FALSE FALSE
2018-06-05 15:40:00 FALSE FALSE FALSE FALSE
2018-07-05 15:20:00 TRUE TRUE TRUE TRUE
2018-08-07 16:16:00 TRUE TRUE TRUE TRUE
2018-09-04 15:50:00 TRUE TRUE TRUE TRUE
2018-10-16 16:03:00 TRUE TRUE TRUE TRUE
2018-10-30 16:07:00 FALSE FALSE FALSE FALSE
2018-12-04 16:42:00 FALSE FALSE FALSE FALSE
2019-01-10 17:10:00 FALSE FALSE FALSE FALSE
2019-02-11 17:55:00 FALSE FALSE FALSE FALSE
2019-03-26 16:10:00 FALSE FALSE FALSE FALSE
2019-05-07 16:10:00 FALSE FALSE FALSE FALSE
2019-06-04 15:25:00 TRUE TRUE TRUE TRUE
2019-07-01 14:53:00 TRUE TRUE TRUE TRUE
2019-08-06 15:55:00 TRUE TRUE TRUE TRUE
2019-09-04 16:00:00 TRUE TRUE TRUE TRUE
2019-10-16 17:05:00 TRUE TRUE TRUE TRUE
2019-12-04 17:40:00 FALSE FALSE FALSE FALSE
2020-01-09 15:12:00 FALSE FALSE FALSE FALSE
2020-02-11 15:50:00 FALSE FALSE FALSE FALSE
2020-03-10 16:15:00 FALSE FALSE FALSE FALSE
2020-06-09 15:35:00 TRUE TRUE TRUE TRUE
2020-06-30 15:22:00 FALSE FALSE FALSE FALSE
2020-08-04 15:07:00 FALSE FALSE FALSE FALSE
2020-09-02 13:27:00 FALSE FALSE FALSE FALSE
2021-01-07 16:08:00 FALSE FALSE FALSE FALSE
2021-02-02 17:15:00 FALSE FALSE FALSE FALSE
2021-03-02 18:05:00 FALSE FALSE FALSE FALSE
2021-04-07 15:45:00 FALSE FALSE FALSE FALSE
2021-05-04 15:22:00 FALSE FALSE FALSE FALSE
aos_datetime check_3day check_1day check_6hr check_1hr
2018-07-05 15:20:00 TRUE TRUE TRUE TRUE
2018-08-07 16:16:00 TRUE TRUE TRUE TRUE
2018-09-04 15:50:00 TRUE TRUE TRUE TRUE
2018-10-16 16:03:00 TRUE TRUE TRUE TRUE
2019-06-04 15:25:00 TRUE TRUE TRUE TRUE
2019-07-01 14:53:00 TRUE TRUE TRUE TRUE
2019-08-06 15:55:00 TRUE TRUE TRUE TRUE
2019-09-04 16:00:00 TRUE TRUE TRUE TRUE
2019-10-16 17:05:00 TRUE TRUE TRUE TRUE
2020-06-09 15:35:00 TRUE TRUE TRUE TRUE

collect_date Fe TOC DOC abs254 abs280
2018-07-05 0.0005 42.81 34.46 0.6168 0.3113
2018-08-07 0.0005 47.99 37.81 0.6288 0.3172
2018-09-04 0.0005 41.88 39.81 0.6569 0.3274
2018-10-16 0.0005 42.74 39.67 0.6455 0.3171
2019-06-04 0.0005 38.82 35.19 0.5498 0.2846
2019-07-01 0.0042 36.23 35.90 0.7863 0.4191
2019-07-01 0.0048 34.88 35.91 0.7881 0.4111
2019-07-01 0.0040 36.78 35.79 0.7881 0.4048
2019-08-06 0.0100 36.05 35.01 0.7538 0.3852
2019-09-04 0.0025 33.08 32.87 0.6865 0.3524
2019-09-04 0.0030 32.76 33.03 0.6967 0.3607
2019-09-04 0.0026 32.82 33.16 0.6503 0.3363
2019-10-16 0.0005 27.63 27.56 0.6416 0.3293
2020-06-09 NA 26.65 26.72 NA NA

Same plot as above with only QF = 0 data

aos_datetime check_3day check_1day check_6hr check_1hr
2014-05-14 15:15:00 FALSE FALSE FALSE FALSE
2014-06-19 16:00:00 FALSE FALSE FALSE FALSE
2016-10-19 15:30:00 FALSE FALSE FALSE FALSE
2016-11-16 16:00:00 FALSE FALSE FALSE FALSE
2016-12-19 17:53:00 FALSE FALSE FALSE FALSE
2017-01-18 16:15:00 FALSE FALSE FALSE FALSE
2017-02-13 16:15:00 FALSE FALSE FALSE FALSE
2017-03-22 14:45:00 FALSE FALSE FALSE FALSE
2017-04-12 15:15:00 FALSE FALSE FALSE FALSE
2017-05-03 14:20:00 FALSE FALSE FALSE FALSE
2017-06-08 14:45:00 FALSE FALSE FALSE FALSE
2017-07-06 15:00:00 FALSE FALSE FALSE FALSE
2017-08-03 14:50:00 FALSE FALSE FALSE FALSE
2017-09-06 14:15:00 FALSE FALSE FALSE FALSE
2017-10-19 15:00:00 FALSE FALSE FALSE FALSE
2017-11-01 15:15:00 FALSE FALSE FALSE FALSE
2018-01-04 17:00:00 FALSE FALSE FALSE FALSE
2018-02-13 16:45:00 FALSE FALSE FALSE FALSE
2018-03-13 16:05:00 FALSE FALSE FALSE FALSE
2018-04-17 16:03:00 FALSE FALSE FALSE FALSE
2018-05-01 16:26:00 FALSE FALSE FALSE FALSE
2018-06-05 15:40:00 FALSE FALSE FALSE FALSE
2018-07-05 15:20:00 FALSE FALSE FALSE FALSE
2018-08-07 16:16:00 FALSE FALSE FALSE FALSE
2018-09-04 15:50:00 FALSE FALSE FALSE FALSE
2018-10-16 16:03:00 FALSE FALSE FALSE FALSE
2018-10-30 16:07:00 FALSE FALSE FALSE FALSE
2018-12-04 16:42:00 FALSE FALSE FALSE FALSE
2019-01-10 17:10:00 FALSE FALSE FALSE FALSE
2019-02-11 17:55:00 FALSE FALSE FALSE FALSE
2019-03-26 16:10:00 FALSE FALSE FALSE FALSE
2019-05-07 16:10:00 FALSE FALSE FALSE FALSE
2019-06-04 15:25:00 FALSE FALSE FALSE FALSE
2019-07-01 14:53:00 TRUE TRUE TRUE TRUE
2019-08-06 15:55:00 TRUE TRUE TRUE TRUE
2019-09-04 16:00:00 TRUE TRUE TRUE TRUE
2019-10-16 17:05:00 TRUE TRUE TRUE TRUE
2019-12-04 17:40:00 FALSE FALSE FALSE FALSE
2020-01-09 15:12:00 FALSE FALSE FALSE FALSE
2020-02-11 15:50:00 FALSE FALSE FALSE FALSE
2020-03-10 16:15:00 FALSE FALSE FALSE FALSE
2020-06-09 15:35:00 FALSE FALSE FALSE FALSE
2020-06-30 15:22:00 FALSE FALSE FALSE FALSE
2020-08-04 15:07:00 FALSE FALSE FALSE FALSE
2020-09-02 13:27:00 FALSE FALSE FALSE FALSE
2021-01-07 16:08:00 FALSE FALSE FALSE FALSE
2021-02-02 17:15:00 FALSE FALSE FALSE FALSE
2021-03-02 18:05:00 FALSE FALSE FALSE FALSE
2021-04-07 15:45:00 FALSE FALSE FALSE FALSE
2021-05-04 15:22:00 FALSE FALSE FALSE FALSE
aos_datetime check_3day check_1day check_6hr check_1hr
2019-07-01 14:53:00 TRUE TRUE TRUE TRUE
2019-08-06 15:55:00 TRUE TRUE TRUE TRUE
2019-09-04 16:00:00 TRUE TRUE TRUE TRUE
2019-10-16 17:05:00 TRUE TRUE TRUE TRUE
## Warning: Removed 641 rows containing missing values (geom_point).

Compare AOS and AIS

Colors indicate month

II.(D) SUNA AIS time series

  • nitrate sensor 15 minute interval time series
  • value is from a burst of readings after lamp warms up
  • error bars on these plots are std error of mean, not expanded uncertainty like for other sensors
flightline_datetime check_30day check_10day check_1day check_12hr
2016-06-29 15:45:47 FALSE FALSE FALSE FALSE
2016-06-29 15:52:09 FALSE FALSE FALSE FALSE
2016-06-29 17:22:24 FALSE FALSE FALSE FALSE
2017-06-21 20:58:28 FALSE FALSE FALSE FALSE
2017-06-26 15:33:21 FALSE FALSE FALSE FALSE
2017-06-26 16:04:51 FALSE FALSE FALSE FALSE
2017-06-26 16:11:11 FALSE FALSE FALSE FALSE
2019-07-26 15:20:23 TRUE TRUE TRUE TRUE
2019-07-26 16:22:11 TRUE TRUE TRUE TRUE
2019-07-26 22:09:25 TRUE TRUE TRUE TRUE
2019-07-27 15:27:04 TRUE TRUE TRUE TRUE
2019-07-30 15:34:20 TRUE TRUE TRUE TRUE
2020-06-24 17:36:27 TRUE TRUE TRUE TRUE
2020-06-24 16:26:15 TRUE TRUE TRUE TRUE
2020-06-24 16:18:44 TRUE TRUE TRUE TRUE
2020-06-26 19:28:06 TRUE TRUE TRUE TRUE
2020-07-02 17:14:10 TRUE TRUE TRUE TRUE
flightline_datetime check_30day check_10day check_1day check_12hr
2019-07-26 15:20:23 TRUE TRUE TRUE TRUE
2019-07-26 16:22:11 TRUE TRUE TRUE TRUE
2019-07-26 22:09:25 TRUE TRUE TRUE TRUE
2019-07-27 15:27:04 TRUE TRUE TRUE TRUE
2019-07-30 15:34:20 TRUE TRUE TRUE TRUE
2020-06-24 17:36:27 TRUE TRUE TRUE TRUE
2020-06-24 16:26:15 TRUE TRUE TRUE TRUE
2020-06-24 16:18:44 TRUE TRUE TRUE TRUE
2020-06-26 19:28:06 TRUE TRUE TRUE TRUE
2020-07-02 17:14:10 TRUE TRUE TRUE TRUE

Helper table for manually pulling SUNA data (AOP dates):

  • Helper table for pulling L0 SUNA for flight dates with SUNA data.
  • Interval is 3 days before and after to facilitate downloads.
  • Connect to VPN and use DQ Blizzard. View Level 0 data, extract L0 sub-stream
  • stream 006 is Absorbance at 254
  • stream 007 is Absorbance at 350
flightline_datetime check_1day interval3_start interval3_end
2019-07-26 15:20:23 TRUE 2019-07-23 2019-07-29
2019-07-26 16:22:11 TRUE 2019-07-23 2019-07-29
2019-07-26 22:09:25 TRUE 2019-07-23 2019-07-29
2019-07-27 15:27:04 TRUE 2019-07-24 2019-07-30
2019-07-30 15:34:20 TRUE 2019-07-27 2019-08-02
2020-06-24 17:36:27 TRUE 2020-06-21 2020-06-27
2020-06-24 16:26:15 TRUE 2020-06-21 2020-06-27
2020-06-24 16:18:44 TRUE 2020-06-21 2020-06-27
2020-06-26 19:28:06 TRUE 2020-06-23 2020-06-29
2020-07-02 17:14:10 TRUE 2020-06-29 2020-07-05
  • Those are the availabilities for the SUNA data at flight times.
  • Save 1 file for each year eg NEON-L0-SUNA/AOP-dates/{mysite}/{mysite}-103-2020.csv
  • Process raw suna data with process-L0-SUNA in processing project

2019 abs plots

## Rows: 803
## Columns: 9
## Delimiter: ","
## dbl  [8]: mean_abs254, sd_abs254, n_abs254, se_abs254, mean_abs350, sd_abs350, n_abs350,...
## dttm [1]: burst_id
## 
## Use `spec()` to retrieve the guessed column specification
## Pass a specification to the `col_types` argument to quiet this message

2019 abs avg

collectDateTime datetime mean_abs254 abs254_ma01 abs254_ma03 abs254_ma04 abs254_ma04u abs254_ma06 abs254_ma12
2019-07-26 15:20:23 2019-07-26 15:15:00 0.7240 0.7223 0.7223 0.7224 0.7246 0.7224 0.7220
2019-07-26 16:22:11 2019-07-26 16:15:00 0.7213 0.7213 0.7220 0.7220 0.7235 0.7220 0.7218
2019-07-26 22:09:25 2019-07-26 22:15:00 0.7213 0.7201 0.7182 0.7182 0.7175 0.7182 0.7182
2019-07-27 15:27:04 2019-07-27 15:30:00 0.6946 0.6944 0.6928 0.6925 0.6904 0.6928 0.6931
2019-07-30 15:34:20 2019-07-30 15:30:00 0.6522 0.6524 0.6523 0.6531 0.6536 0.6560 0.6577
collectDateTime datetime mean_abs350 abs350_ma01 abs350_ma03 abs350_ma04 abs350_ma04u abs350_ma06 abs350_ma12
2019-07-26 15:20:23 2019-07-26 15:15:00 0.1961 0.1945 0.1945 0.1945 0.1966 0.1945 0.1938
2019-07-26 16:22:11 2019-07-26 16:15:00 0.1936 0.1936 0.1936 0.1936 0.1954 0.1936 0.1931
2019-07-26 22:09:25 2019-07-26 22:15:00 0.1912 0.1905 0.1903 0.1903 0.1887 0.1903 0.1902
2019-07-27 15:27:04 2019-07-27 15:30:00 0.1660 0.1659 0.1654 0.1653 0.1636 0.1654 0.1655
2019-07-30 15:34:20 2019-07-30 15:30:00 0.1245 0.1245 0.1243 0.1258 0.1269 0.1278 0.1315

2020 abs plots

## Rows: 755
## Columns: 9
## Delimiter: ","
## dbl  [8]: mean_abs254, sd_abs254, n_abs254, se_abs254, mean_abs350, sd_abs350, n_abs350,...
## dttm [1]: burst_id
## 
## Use `spec()` to retrieve the guessed column specification
## Pass a specification to the `col_types` argument to quiet this message

2020 abs avg

collectDateTime datetime mean_abs254 abs254_ma01 abs254_ma03 abs254_ma04 abs254_ma04u abs254_ma06 abs254_ma12
2020-06-24 17:36:27 2020-06-24 17:30:00 0.4602 0.4606 0.4606 0.4607 0.4609 0.4609 0.4611
2020-06-24 16:26:15 2020-06-24 16:30:00 0.4600 0.4600 0.4602 0.4604 0.4608 0.4609 0.4611
2020-06-24 16:18:44 2020-06-24 16:15:00 0.4598 0.4600 0.4602 0.4604 0.4609 0.4608 0.4612
2020-06-26 19:28:06 2020-06-26 19:30:00 0.4726 0.4727 0.4727 0.4727 0.4733 0.4727 0.4732
collectDateTime datetime mean_abs350 abs350_ma01 abs350_ma03 abs350_ma04 abs350_ma04u abs350_ma06 abs350_ma12
2020-06-24 17:36:27 2020-06-24 17:30:00 0.0449 0.0450 0.0452 0.0452 0.0455 0.0457 0.0456
2020-06-24 16:26:15 2020-06-24 16:30:00 0.0449 0.0447 0.0448 0.0450 0.0453 0.0455 0.0457
2020-06-24 16:18:44 2020-06-24 16:15:00 0.0443 0.0447 0.0447 0.0450 0.0453 0.0454 0.0458
2020-06-26 19:28:06 2020-06-26 19:30:00 0.0545 0.0548 0.0548 0.0548 0.0550 0.0548 0.0554

x2019 SUNA plots 2019

  • filtered to no Quality Flags

x2020 SUNA plots 2020

  • Not filtered by QFs
  • QF because of lack of calibration?

II.(E) SUNA vs AOS absorbs

  • Use surface water chemistry data to see date and times of ground sample collection for UV absorbance data

This is the availability of SUNA data for AOS sampling dates

aos_datetime check_3day check_1day check_6hr check_1hr
2016-10-19 15:30:00 FALSE FALSE FALSE FALSE
2016-11-16 16:00:00 FALSE FALSE FALSE FALSE
2016-12-19 17:53:00 FALSE FALSE FALSE FALSE
2017-01-18 16:15:00 FALSE FALSE FALSE FALSE
2017-02-13 16:15:00 FALSE FALSE FALSE FALSE
2017-03-22 14:45:00 FALSE FALSE FALSE FALSE
2017-04-12 15:15:00 FALSE FALSE FALSE FALSE
2017-05-03 14:20:00 FALSE FALSE FALSE FALSE
2017-06-08 14:45:00 FALSE FALSE FALSE FALSE
2017-07-06 15:00:00 FALSE FALSE FALSE FALSE
2017-08-03 14:50:00 FALSE FALSE FALSE FALSE
2017-09-06 14:15:00 FALSE FALSE FALSE FALSE
2017-10-19 15:00:00 FALSE FALSE FALSE FALSE
2017-11-01 15:15:00 FALSE FALSE FALSE FALSE
2018-01-04 17:00:00 FALSE FALSE FALSE FALSE
2018-02-13 16:45:00 FALSE FALSE FALSE FALSE
2018-03-13 16:05:00 FALSE FALSE FALSE FALSE
2018-04-17 16:03:00 FALSE FALSE FALSE FALSE
2018-05-01 16:26:00 FALSE FALSE FALSE FALSE
2018-06-05 15:40:00 FALSE FALSE FALSE FALSE
2018-07-05 15:20:00 FALSE FALSE FALSE FALSE
2018-08-07 16:16:00 FALSE FALSE FALSE FALSE
2018-09-04 15:50:00 FALSE FALSE FALSE FALSE
2018-10-16 16:03:00 FALSE FALSE FALSE FALSE
2018-10-30 16:07:00 FALSE FALSE FALSE FALSE
2018-12-04 16:42:00 FALSE FALSE FALSE FALSE
2019-01-10 17:10:00 FALSE FALSE FALSE FALSE
2019-02-11 17:55:00 FALSE FALSE FALSE FALSE
2019-03-26 16:10:00 FALSE FALSE FALSE FALSE
2019-05-07 16:10:00 FALSE FALSE FALSE FALSE
2019-06-04 15:25:00 FALSE FALSE FALSE FALSE
2019-07-01 14:53:00 TRUE TRUE TRUE TRUE
2019-08-06 15:55:00 TRUE TRUE TRUE TRUE
2019-09-04 16:00:00 TRUE TRUE TRUE TRUE
2019-10-16 17:05:00 TRUE TRUE TRUE TRUE
2019-12-04 17:40:00 FALSE FALSE FALSE FALSE
2020-08-04 15:07:00 FALSE FALSE FALSE FALSE
2020-09-02 13:27:00 FALSE FALSE FALSE FALSE
2021-01-07 16:08:00 FALSE FALSE FALSE FALSE
2021-02-02 17:15:00 FALSE FALSE FALSE FALSE
2021-03-02 18:05:00 FALSE FALSE FALSE FALSE
2021-04-07 15:45:00 TRUE FALSE FALSE FALSE
2021-05-04 15:22:00 TRUE TRUE TRUE TRUE
aos_datetime check_3day check_1day check_6hr check_1hr
2019-07-01 14:53:00 TRUE TRUE TRUE TRUE
2019-08-06 15:55:00 TRUE TRUE TRUE TRUE
2019-09-04 16:00:00 TRUE TRUE TRUE TRUE
2019-10-16 17:05:00 TRUE TRUE TRUE TRUE
2021-04-07 15:45:00 TRUE FALSE FALSE FALSE
2021-05-04 15:22:00 TRUE TRUE TRUE TRUE

Helper table for manually pulling SUNA data (AOS dates):

aos_datetime check_1day interval3_start interval3_end
2019-07-01 14:53:00 TRUE 2019-06-28 2019-07-04
2019-08-06 15:55:00 TRUE 2019-08-03 2019-08-09
2019-09-04 16:00:00 TRUE 2019-09-01 2019-09-07
2019-10-16 17:05:00 TRUE 2019-10-13 2019-10-19
2021-04-07 15:45:00 FALSE 2021-04-04 2021-04-10
2021-05-04 15:22:00 TRUE 2021-05-01 2021-05-07
  • processed SUNA absorbance data are in box folder NEON-processed/suna-L0-timeseries
  • how do absorbances from nitrate sensor compare to AOS samples?

Regression for sensor data within the same day

## `geom_smooth()` using formula 'y ~ x'

III. Suspended Matter

III.(A) AOS Sampling

  • From processed Surface water chemistry files on Box
  • TDS passes through .45 micron filter
  • TSS is from weight of dried mass that doesnt pass through filter

Data from all locations at site

## geom_path: Each group consists of only one observation. Do you need to adjust
## the group aesthetic?
## geom_path: Each group consists of only one observation. Do you need to adjust
## the group aesthetic?
## geom_path: Each group consists of only one observation. Do you need to adjust
## the group aesthetic?

sm_site_df <- sm_site_df %>%
  dplyr::filter(namedLocation %in% my_loc)
## Warning: Transformation introduced infinite values in continuous y-axis

## Warning: Transformation introduced infinite values in continuous y-axis

How much does TDS/TSS change?

  • Adjusts sample IDs in a new column to group together raw and filtered, while keeping track of duplicates
  • Dissolved passes through .45 micron filter, suspended does not
## Warning: Removed 1 row(s) containing missing values (geom_path).
## Warning: Removed 2 rows containing missing values (geom_point).

Match sediment aos to aop dates

source('R/match-aop-aos.R')

sm_match_list <- match_aop_aos(mysite_dates, sm_site_df, 'collect_date')

knitr::kable(sm_match_list$dates)
flightdate aos_before aos_after days_before days_after min_days meets_thresh1 meets_thresh2
2016-06-29 2014-06-19 2016-10-19 741 112 112 FALSE FALSE
2017-06-21 2017-06-08 2017-07-06 13 15 13 TRUE TRUE
2017-06-26 2017-06-08 2017-07-06 18 10 10 TRUE TRUE
2019-07-26 2019-07-01 2019-08-06 25 11 11 TRUE TRUE
2019-07-27 2019-07-01 2019-08-06 26 10 10 TRUE TRUE
2019-07-30 2019-07-01 2019-08-06 29 7 7 TRUE TRUE
2020-06-24 2019-12-04 2021-05-04 203 314 203 FALSE FALSE
2020-06-26 2019-12-04 2021-05-04 205 312 205 FALSE FALSE
2020-07-02 2019-12-04 2021-05-04 211 306 211 FALSE FALSE
knitr::kable(sm_match_list$matches)
flightdate aos_match days
2017-06-21 2017-06-08 13
2017-06-26 2017-07-06 10
2019-07-26 2019-08-06 11
2019-07-27 2019-08-06 10
2019-07-30 2019-08-06 7

TSS x flightdate

These are the closest in time ground-based measurements of TDS (mg/L) and TSS (mg/L)

  • List of values means there are duplicate samples (multiple sampleIDs) from same date.
aos_match flightdate days TDS TSS TSS - Dry Mass
2017-06-08 2017-06-21 13 910.1 32 800
2017-07-06 2017-06-26 10 885.62, 887.64, 891.23 10, 9, 9 900, 900, 1000
2019-08-06 2019-07-26 11 744.6 10 500
2019-08-06 2019-07-27 10 744.6 10 500
2019-08-06 2019-07-30 7 744.6 10 500

III.(B) Turbidity AIS time series

The EXO turbidity sensor employs a near‐IR light source (~780 ‐ 900 nm) and detects scattering at 90 degrees of the incident beam.

  • processed turbidity water quality time series on box
flightline_datetime check_30day check_10day check_1day check_12hr
2016-06-29 15:45:47 FALSE FALSE FALSE FALSE
2016-06-29 15:52:09 FALSE FALSE FALSE FALSE
2016-06-29 17:22:24 FALSE FALSE FALSE FALSE
2017-06-21 20:58:28 FALSE FALSE FALSE FALSE
2017-06-26 15:33:21 FALSE FALSE FALSE FALSE
2017-06-26 16:04:51 FALSE FALSE FALSE FALSE
2017-06-26 16:11:11 FALSE FALSE FALSE FALSE
2019-07-26 15:20:23 TRUE TRUE TRUE TRUE
2019-07-26 16:22:11 TRUE TRUE TRUE TRUE
2019-07-26 22:09:25 TRUE TRUE TRUE TRUE
2019-07-27 15:27:04 TRUE TRUE TRUE TRUE
2019-07-30 15:34:20 TRUE TRUE TRUE TRUE
2020-06-24 17:36:27 TRUE TRUE TRUE TRUE
2020-06-24 16:26:15 TRUE TRUE TRUE TRUE
2020-06-24 16:18:44 TRUE TRUE TRUE TRUE
2020-06-26 19:28:06 TRUE TRUE TRUE FALSE
2020-07-02 17:14:10 TRUE TRUE FALSE FALSE

2019 Turbidity plots

  • filtered to no quality flags

2020 Turbidity plots

  • Not filtered for quality flags
  • QF because of lack of calibration?
## Warning: Removed 8537 row(s) containing missing values (geom_path).
## Warning: Removed 8658 rows containing missing values (geom_point).

## Warning: Removed 2438 row(s) containing missing values (geom_path).
## Warning: Removed 2552 rows containing missing values (geom_point).

## Warning: Removed 444 row(s) containing missing values (geom_path).
## Warning: Removed 444 rows containing missing values (geom_point).

## Warning: Removed 4737 row(s) containing missing values (geom_path).
## Warning: Removed 4737 rows containing missing values (geom_point).

2019 turb avg

  • Smooth the fdom time series using a rolling mean with zoo::rollapply.
  • First convert to a regular 5 minute time series using tsibble

Get the moving average value closet to flight time.

  • 5min is the original data from that row
  • ma01 = 1 hour window, ma03 = 3 hour window, and so on
  • chl_ma04u is mean, the others are all mean
collectDateTime datetime turb5min ma01 ma03 ma04 ma04u ma06 ma12
2019-07-26 15:20:23 2019-07-26 15:20:00 2.09 2.170 2.170 2.175 2.228913 2.180 2.240
2019-07-26 16:22:11 2019-07-26 16:20:00 2.17 2.205 2.230 2.240 2.326667 2.245 2.290
2019-07-26 22:09:25 2019-07-26 22:10:00 3.98 3.185 3.140 3.125 3.129167 3.080 2.925
2019-07-27 15:27:04 2019-07-27 15:25:00 3.62 2.985 2.930 2.910 2.942917 2.790 2.600
2019-07-30 15:34:20 2019-07-30 15:35:00 2.75 2.770 2.795 2.820 2.921591 2.820 2.820

2020 turb avg

  • Smooth the fdom time series using a rolling mean with zoo::rollapply.
  • First convert to a regular 5 minute time series using tsibble

Get the moving average value closet to flight time.

  • 5min is the original data from that row
  • ma01 = 1 hour window, ma03 = 3 hour window, and so on
  • chl_ma04u is mean, the others are all mean
collectDateTime datetime turb5min ma01 ma03 ma04 ma04u ma06 ma12
2020-06-24 17:36:27 2020-06-24 17:35:00 1.91 1.865 1.820 1.865 3.284375 1.845 1.9
2020-06-24 16:26:15 2020-06-24 16:25:00 1.51 1.795 1.875 1.840 2.687708 1.875 1.9
2020-06-24 16:18:44 2020-06-24 16:20:00 1.93 1.795 1.875 1.815 2.677292 1.875 1.9
2020-06-26 19:28:06 2020-06-26 19:30:00 NA NA NA NA NaN NA NA
2020-07-02 17:14:10 2020-07-02 17:15:00 NA NA NA NA NaN NA NA

III.(C) Turbidity vs TDS/ TSS (TODO)

  • use swchem_df date and times with the aop/ais matching function to determine which sampling dates have sensor data from same day around time of sampling.

  • Get fDOM sensor data from all AOS sampling dates (use ais with buffer2 function?)

  • show what is available

  • calculate rolling averages of fDOM time series

  • get rolling average values for time closest to sampling using which min on difftime

  • little overlap because of seasonal removal of buoys

aos_datetime check_3day check_1day check_6hr check_1hr
2014-05-14 15:15:00 FALSE FALSE FALSE FALSE
2014-06-19 16:00:00 FALSE FALSE FALSE FALSE
2016-10-19 15:30:00 FALSE FALSE FALSE FALSE
2016-11-16 16:00:00 FALSE FALSE FALSE FALSE
2016-12-19 17:53:00 FALSE FALSE FALSE FALSE
2017-01-18 16:15:00 FALSE FALSE FALSE FALSE
2017-02-13 16:15:00 FALSE FALSE FALSE FALSE
2017-03-22 14:45:00 FALSE FALSE FALSE FALSE
2017-04-12 15:15:00 FALSE FALSE FALSE FALSE
2017-05-03 14:20:00 FALSE FALSE FALSE FALSE
2017-06-08 14:45:00 FALSE FALSE FALSE FALSE
2017-07-06 15:00:00 FALSE FALSE FALSE FALSE
2017-08-03 14:50:00 FALSE FALSE FALSE FALSE
2017-09-06 14:15:00 FALSE FALSE FALSE FALSE
2017-10-19 15:00:00 FALSE FALSE FALSE FALSE
2017-11-01 15:15:00 FALSE FALSE FALSE FALSE
2018-01-04 17:00:00 FALSE FALSE FALSE FALSE
2018-02-13 16:45:00 FALSE FALSE FALSE FALSE
2018-03-13 16:05:00 FALSE FALSE FALSE FALSE
2018-04-17 16:03:00 FALSE FALSE FALSE FALSE
2018-05-01 16:26:00 FALSE FALSE FALSE FALSE
2018-06-05 15:40:00 FALSE FALSE FALSE FALSE
2018-07-05 15:20:00 TRUE TRUE TRUE TRUE
2018-08-07 16:16:00 TRUE TRUE TRUE TRUE
2018-09-04 15:50:00 TRUE TRUE TRUE TRUE
2018-10-16 16:03:00 TRUE TRUE TRUE TRUE
2018-10-30 16:07:00 FALSE FALSE FALSE FALSE
2018-12-04 16:42:00 FALSE FALSE FALSE FALSE
2019-01-10 17:10:00 FALSE FALSE FALSE FALSE
2019-02-11 17:55:00 FALSE FALSE FALSE FALSE
2019-03-26 16:10:00 FALSE FALSE FALSE FALSE
2019-05-07 16:10:00 FALSE FALSE FALSE FALSE
2019-06-04 15:25:00 TRUE TRUE TRUE TRUE
2019-07-01 14:53:00 TRUE TRUE TRUE TRUE
2019-08-06 15:55:00 TRUE TRUE TRUE TRUE
2019-09-04 16:00:00 TRUE TRUE TRUE TRUE
2019-10-16 17:05:00 TRUE TRUE TRUE TRUE
2019-12-04 17:40:00 FALSE FALSE FALSE FALSE
2021-05-04 15:22:00 FALSE FALSE FALSE FALSE
aos_datetime check_3day check_1day check_6hr check_1hr
2018-07-05 15:20:00 TRUE TRUE TRUE TRUE
2018-08-07 16:16:00 TRUE TRUE TRUE TRUE
2018-09-04 15:50:00 TRUE TRUE TRUE TRUE
2018-10-16 16:03:00 TRUE TRUE TRUE TRUE
2019-06-04 15:25:00 TRUE TRUE TRUE TRUE
2019-07-01 14:53:00 TRUE TRUE TRUE TRUE
2019-08-06 15:55:00 TRUE TRUE TRUE TRUE
2019-09-04 16:00:00 TRUE TRUE TRUE TRUE
2019-10-16 17:05:00 TRUE TRUE TRUE TRUE

collect_date TSS TDS
2018-07-05 4 945.20
2018-08-07 6 935.64
2018-09-04 10 977.41
2018-10-16 34 1053.00
2019-06-04 8 872.15
2019-07-01 6 921.41
2019-07-01 8 915.63
2019-07-01 6 918.42
2019-08-06 10 744.60
2019-09-04 20 950.23
2019-09-04 18 942.21
2019-09-04 20 958.42
2019-10-16 12 833.27

Same plot as above with only QF = 0 data

aos_datetime check_3day check_1day check_6hr check_1hr
2014-05-14 15:15:00 FALSE FALSE FALSE FALSE
2014-06-19 16:00:00 FALSE FALSE FALSE FALSE
2016-10-19 15:30:00 FALSE FALSE FALSE FALSE
2016-11-16 16:00:00 FALSE FALSE FALSE FALSE
2016-12-19 17:53:00 FALSE FALSE FALSE FALSE
2017-01-18 16:15:00 FALSE FALSE FALSE FALSE
2017-02-13 16:15:00 FALSE FALSE FALSE FALSE
2017-03-22 14:45:00 FALSE FALSE FALSE FALSE
2017-04-12 15:15:00 FALSE FALSE FALSE FALSE
2017-05-03 14:20:00 FALSE FALSE FALSE FALSE
2017-06-08 14:45:00 FALSE FALSE FALSE FALSE
2017-07-06 15:00:00 FALSE FALSE FALSE FALSE
2017-08-03 14:50:00 FALSE FALSE FALSE FALSE
2017-09-06 14:15:00 FALSE FALSE FALSE FALSE
2017-10-19 15:00:00 FALSE FALSE FALSE FALSE
2017-11-01 15:15:00 FALSE FALSE FALSE FALSE
2018-01-04 17:00:00 FALSE FALSE FALSE FALSE
2018-02-13 16:45:00 FALSE FALSE FALSE FALSE
2018-03-13 16:05:00 FALSE FALSE FALSE FALSE
2018-04-17 16:03:00 FALSE FALSE FALSE FALSE
2018-05-01 16:26:00 FALSE FALSE FALSE FALSE
2018-06-05 15:40:00 FALSE FALSE FALSE FALSE
2018-07-05 15:20:00 TRUE TRUE TRUE TRUE
2018-08-07 16:16:00 TRUE TRUE TRUE TRUE
2018-09-04 15:50:00 TRUE TRUE TRUE TRUE
2018-10-16 16:03:00 TRUE TRUE TRUE TRUE
2018-10-30 16:07:00 FALSE FALSE FALSE FALSE
2018-12-04 16:42:00 FALSE FALSE FALSE FALSE
2019-01-10 17:10:00 FALSE FALSE FALSE FALSE
2019-02-11 17:55:00 FALSE FALSE FALSE FALSE
2019-03-26 16:10:00 FALSE FALSE FALSE FALSE
2019-05-07 16:10:00 FALSE FALSE FALSE FALSE
2019-06-04 15:25:00 TRUE TRUE TRUE TRUE
2019-07-01 14:53:00 TRUE TRUE TRUE TRUE
2019-08-06 15:55:00 TRUE TRUE TRUE TRUE
2019-09-04 16:00:00 TRUE TRUE TRUE TRUE
2019-10-16 17:05:00 TRUE TRUE TRUE TRUE
2019-12-04 17:40:00 FALSE FALSE FALSE FALSE
2021-05-04 15:22:00 FALSE FALSE FALSE FALSE
aos_datetime check_3day check_1day check_6hr check_1hr
2018-07-05 15:20:00 TRUE TRUE TRUE TRUE
2018-08-07 16:16:00 TRUE TRUE TRUE TRUE
2018-09-04 15:50:00 TRUE TRUE TRUE TRUE
2018-10-16 16:03:00 TRUE TRUE TRUE TRUE
2019-06-04 15:25:00 TRUE TRUE TRUE TRUE
2019-07-01 14:53:00 TRUE TRUE TRUE TRUE
2019-08-06 15:55:00 TRUE TRUE TRUE TRUE
2019-09-04 16:00:00 TRUE TRUE TRUE TRUE
2019-10-16 17:05:00 TRUE TRUE TRUE TRUE
## Warning: Removed 1279 rows containing missing values (geom_point).

Compare AOS and AIS

Colors indicate month